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Final  Report  on  AFOSR  grant  FA09550-12-1-0183 


Abstract: 

This  research  effort  belongs  to  the  general  area  of  distributed  learning,  and  the  control 
and  coordination  of  multiple  and  single  agent  systems,  and  is  led  by  Dr.  Martinez  and  Dr. 
Krstic  at  UC  San  Diego.  The  work  has  particular  impact  on  the  development  of  a  theory 
of  autonomy  science,  which  accounts  for  both  the  aspects  of  resilience  under  adversaries, 
and  learning  via  extremum  seeking,  and  distributed  optimization  techniques.  During  this 
period,  several  theoretical  advancements  as  well  as  applications  have  been  carried  out, 
which  are  summarized  as  follows. 


Research  Results: 

Dr.  Martinez’s  effort  focuses  on  the  analysis  and  design  of  robust  (multi-)agent 
coordination  algorithms  subject  to  failure  and  adversarial  action.  The  agents  can  represent 
miscellaneous  autonomous  and  semi-autonomous  vehicles  that  are  remotely  controlled  by 
operators.  These  groups  can  be  subject  to  attacks  from  other  external  agents  leading  to 
complex  networked  adversarial  settings.  Attacks  and  malicious  actions  considered 
include  Denial-of-Service  (DoS),  replay,  and  man  in  the  middle  attacks,  eavesdropping 
and  privacy  violation,  and,  more  recently,  virus  spreading.  In  addition  to  this,  work  to 
advance  the  state  of  the  art  in  multi-agent  systems  has  been  continued. 

Technical  Results: 

1.  A  set  of  novel  algorithms  based  on  receding-horizon  control  and  distributed 
parameter  learning  for  the  robust  coordination  of  multi-agent  systems.  A  study  of 
the  tradeoffs  in  costs/effort  of  the  algorithms  under  the  following  set  of 
assumptions  have  been  considered:  (a)  unknown  adversaries  introducing  on/off 
disturbances  to  the  system  (e.g.  denial  of  service),  (b)  partially  known  adversary, 
which  modifies  the  control  commands  to  a  certain  extent;  however,  the  operator 
can  exploit  this  model  to  learn  the  adversary,  (c)  the  adversarial  action  aims  to 
reconstruct  the  whole  multi-agent  system  objective  and  control  commands. 

2.  Novel  time-triggered,  event-triggered,  and  receding-horizon-based  control  laws 
for  remote  system  operation  under  DoS  pulse-width  modulated  systems.  By  means 
of  parameter  tuning  that  learns  the  characteristics  of  the  DoS  signal,  event¬ 
triggering  and  time-triggering  controls  for  a  linear  system  are  derived. 

3.  Privacy  preserving  coordination  algorithms  for  multi-agent  systems.  Consensus 
algorithms  that  do  not  require  agents  to  exchange  state  information,  but  secondary 
variables  which  do  not  allow  for  the  reconstruction  of  the  agents’  states  by  an 
eavesdropping  agent,  are  devised. 

4.  Robust  coordination  (consensus)  of  multi-agent  systems  when  these  are  subject  to 
outlier  measurements.  This  is  based  on  dynamic  consensus  algorithms  combined 
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with  RANSAC,  which  is  a  ‘voting  strategy’  widely  used  in  computer  vision  and 
that  allows  to  deal  with  more  than  50%  outliers. 

5.  Coordination  (consensus  and  distributed  optimization)  of  multi-agent  systems 
using  event-triggered  control.  We  studied  problems  of  dynamic  average 
consensus  and  a  class  of  unconstrained  continuous-time  optimization  algorithms 
for  the  coordination  of  multiple  agents. 

6.  Virus  spreading  mitigation  distributed  optimization  algorithms  over 
computer/robotic  networks.  We  have  produced  a  variety  of  discrete-time  and 
distributed  optimization  algorithms  that  can  be  used  by  a  multi-operator  group  to 
decide  on  an  optimal  resource  allocation  subject  to  constraints  and  eradicate  a 
virus  spreading  over  a  network.  These  algorithms  have  various  robust  features 
(such  as  robustness  to  initialization.) 

We  mention  the  following  work  under  review,  on  subjects  that  are  more  related  to  the 
general  topics  of  distributed  learning,  and  optimization,  in  GPS  denied  environments: 

1.  Distributed  and  discrete-time  convex  optimization  algorithms  subject  for  unknown 
cost  functions.  This  work,  which  is  to  appear,  is  similar  to  simultaneous 
perturbation  stochastic  approximation  (SPSA),  but  based  on  recent  theory  of 
stochastic  hybrid  systems,  and  building  on  the  discrete -time  optimization 
algorithms  of  (6.)  above.  Thus,  it  does  not  have  the  limitations  of  SPSA  such  as 
diminishing  step  sizes. 

2.  Control  of  single  agents  in  GPS  denied  environments  with  obstacles  but  via  an 
adaptation  of  the  SPSA  techniques.  Similar  techniques  as  above  can  be  applied 
for  a  wheeled-robot  to  reach  the  maximum  of  a  convex  cost  function  using 
projections  to  handle  obstacles. 


We  report  33  publications  refereed  publications  resulting  from  the  grant  for  the  period  of 
performance;  see  the  file  ‘publications-afosr.pdf  for  more  information. 


Over  the  period  of  this  grant,  Co-PI  Dr.  Krstic  has  achieved  several  advances  in 
extremum  seeking  described  as  follows: 

Applications: 

1 .  Extremum  seeking  has  been  established  as  one  of  the  powerful  methodologies  for 
navigating  autonomous  vehicles  in  GPS-denied  environments  and  several 
publications  are  reported  below  where  advances  in  “source  seeking”  for  single 
vehicles  and  groups  of  vehicles  are  presented.  For  example,  an  application  to 
GPS-denied  swarms  of  autonomous  vehicles  has  been  developed  using  stochastic 
extremum  seeking,  where  the  vehicles,  starting  from  distinct  and  arbitrary 
positions,  congregate  to  the  source,  while  being  fed  a  common  ( indistinct )  control 
input. 

2.  Maximization  of  UAV  endurance  via  using  air  turbulence  with  extremum 
seekings  is  also  achieved  in  one  of  the  publications  below.  Extremum  seeking  has 
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also  been  demonstrated  on  maximum  power  point  tracking  for  photovoltaic  arrays 
and  for  wind  turbines. 

3.  ES  has  recently  been  implemented  on  the  Mars  Rover  Curiosity  and,  as  of  May 
2015,  is  used  for  the  focusing  of  the  ChemCam  instrument  which  perfonns 
chemical  testing  of  rocks  at  a  distance  by  shining  a  laser  beam,  vaporizing,  the 
rock  surface,  and  focusing  the  reflection  using  a  mirror  back  to  a  camera,  with  the 
help  of  extremum  seeking,  with  the  image  subsequently  being  used  in  the 
chemical  analysis  of  the  rock. 

4.  Extremum  seeking  has  also  been  implemented  as  the  enabling  technology  in 
extreme  ultraviolet  light  sources  on  ASML-Cymer  silicon  wafer  scanners.  This 
technology  reduces  the  feature  sizes  of  transistors  on  microchips  by  4-10  times 
compared  to  previous  technologies.  Eight  such  units,  at  the  price  of  $100  million 
per  unit,  have  been  delivered  to  semiconductor  fabs  in  Taiwan  and  China  and 
running  since  2014. 

5.  Extremum  seeking  has  also  been  implemented  on  a  kilometer-long  accelerator  at 
the  Los  Alamos  National  Laboratory  for  optimizing  the  high  voltage  converter 
modulator. 

Theory:  A  number  of  theoretical  advances  in  extremum  seeking  are  reported  in  the 
publications  listed  below. 

1.  Extensions  from  deterministic  to  stochastic  extremum  seeking  in  several  papers 
and  one  book. 

2.  Extension  of  ES  from  single-agent  optimization  to  non-cooperative  games  in 
which  the  payoff  functions  are  unknown  to  the  players.  ES  performs  the 
optimization  of  the  players’  strategies  with  a  guaranteed  convergence  to  the  Nash 
equilibrium  strategies. 

3.  Extension  from  gradient-like  to  Newton-like  optimization  strategies. 

4.  Development  of  ES  algorithms  for  batch-to-batch  optimization  of  open-loop 
finite-horizon  optimal  control  strategies  (which  has  been  applied  to  lasers  and  in 
charged  particle  accelerators). 

5.  Development  of  model-free  feedback  algorithms  for  stabilization  of  nonlinear 
systems  using  the  application  of  ES  with  candidate  control  Lyapunov  functions. 

Dr.  Krstic  reports  about  50  peer-reviewed  publications  resulting  from  the  grant  for  the 
period  of  perfonnance.  See  the  file  ‘publications-afors.pdf  for  more  infonnation. 


Next  we  give  recognitions  earned  by  Martinez  and  Krstic  during  this  period. 

AWARDS  AND  HONORS  (Dr.  Martinez) 

Career  promotions  and  recognitions 
2014  Full  Professor,  UC  San  Diego 

2016  Senior  Editor,  IEEE  Transactions  on  Control  of  Networked  Systems 
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Awards  to  Students 

201 1  Minghui  Zhu,  2011  MAE  Outstanding  Graduate  Student  of  the  Year  award,  now  an 
Assistant  Professor  at  Penn  State  University 

2013  Solmaz  S.  Kia,  UC  Presidential  Fellowship  recipient,  now  an  Assistant  Professor  at 
UC  Irvine 

2015  Andres  Cortes,  2015  MAE  Outstanding  Graduate  Student  of  the  Year  award,  now  an 
Assistant  Professor  at  Penn  State  University,  now  at  EPRI 

KEYNOTE  LECTURES 

2016  Keynote  Speaker  at  the  “ Workshop  on  Control  and  Observability  of  Network 
Dynamics,”  Mathematical  Biosciences  Institute,  Ohio  State  University,  April  2016 

2016  Keynote  Speaker  at  the  “1st  So-Cal  Robotics  Symposium,”  Mathematical  Biosciences 
Institute,  Ohio  State  University,  April  2016 

2013  Keynote  Speaker  at  the  “Is'  Workshop  on  Wireless  Intelligent  Sensor  Networks 
(WISeNet),”  Duke  University,  2013 

2012  Plenary  Speaker  at  the  “3rd  IF  AC  Workshop  on  Distributed  Estimation  and  Control  of 
Networked  Systems”  (NECSYS),  Santa  Barbara,  2012 


AWARDS  AND  HONORS  (Dr.  Krstic) 

Career  recognitions 

2015  Foreign  Member,  Academy  of  Engineering  of  Serbia 

2015  ASME  DSCD  Nyquist  Lecture  Prize 

2015  Fellow  of  SIAM  (Society  for  Industrial  and  Applied  Mathematics) 

2015  Associate  Fellow  of  AIAA  (American  Institute  for  Aeronautics  and  Astronautics) 

2014  Fellow  of  ASME  (American  Society  of  Mechanical  Engineers) 

2014  Fellow  o\'  I ET  (Institution  of  Engineering  and  Technology,  UK) 

Other  major  recognitions 

2016  Qian  Ren  Professor,  Northeastern  University,  China 

2015  Invitation  Fellow,  Japan  Society  for  the  Promotion  of  Science 

2015  Honorary  Professor,  Dalian  Maritime  University,  Donghua  University,  Chongqing 
University,  Northeastern  University,  China 

2013  Distinguished  Visiting  Fellow  of  the  Royal  Academy  of  Engineering,  United  Kingdom 

Book  Prize 

2014  Harold  Chestnut  Textbook  Prize,  IFAC  (International  Federation  for  Automatic 
Control),  triennial  prize — the  highest  recognition  for  authoring  a  book  in  the  field  of 
control  systems 

Awards  to  Students 

2014  Chancellor  ’ s  Dissertation  Medal  in  Engineering  (among  150  PhDs  granted  in  2013), 

UCSD,  Nikolaos  Bekiaris-Liberis 

2014  Best  European  Dissertation  in  Control  of  Complex  and  Heterogeneous  Systems, 
European  Embedded  Control  Institute  (EECI),  Delphine  Bresch-Pietri 

2013  Best  University-wide  Dissertation  Award,  Paris-Tech  (includes  12  most  prestigious 
and  oldest  technical  and  business  universities  in  the  Paris  region  such  as  Ecole  des 
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Mines,  Ecole  Poly  technique,  ENSAE,  etc.),  Delphine  Bresch-Pietri 


KEYNOTE  LECTURES 

2015  ASME  Dynamic  Systems  and  Control  Conference,  Nyquist  Lecture ,  Columbus,  OH 

2015  IF  AC  Workshop  on  Aclv.  Control  and  Navigation  for  Autonomous  Aerospace  Vehicles, 
Sevilla,  Spain 

2014  IEEE  Conference  on  Decision  and  Control,  Los  Angeles 

2013  Congreso  Nacional  cle  Control  Automatico,  Mexico 

2013  IF  AC  Workshop  on  Control  of  Systems  Governed  by  Partial  Dijf.  Equations,  Paris 

2013  IF  AC  Nonlinear  Control  Systems  Symposium  (NOLCOS),  Toulouse,  France 

2013  IF  AC  Workshop  on  Adaptation  and  Learning  in  Control  and  Signal  Proc.,  Caen,  France 

2013  IEEE  Workshop  on  Open  Problems  and  Challenges  in  Automotive  Control,  Washington  DC 

2012  IEEE  Conference  on  Industrial  Electronics  and  Applications ,  Singapore 

2012  IEEE  International  Conference  on  Mechatronics  and  Automation,  Chengdu,  China 

FELICITATION  LECTURES 

2016  Cor  on  Fest,  UPMC,  Sorbonne  Universites,  Paris 

2015  Praly  Fest,  Ecole  des  Mines,  Paris 
2012  Spong  Fest,  University  of  Texas,  Dallas 
2012  Sontag  Fest,  Rutgers  University 

2012  Bastin  Fest,  Universite  Catholique  de  Louvain,  Belgium 
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Journal  papers 
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